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This  article  describes  how  the  309th  S  oftware  Maintenance  Group  (SMXG)  atHillAFB,  Utah,  used  Standard  Gap  ability 
Maturiy  Model®  Integration  Appraisal  Method  for  Process  Improvement  (SGAMPI)  B  appraisals  as  means  to  identify 
value-added  process  improvements,  educate  key  project  personnel  in  the  organisation  on  best  practices,  and  obtain  critical  buy- 
in  from  project  personnel  within  the  framework  of  an  organisational  transition  from  use  of  the  Gapability  Maturity  Model 
(GMM)  to  the  Gapability  Maturity  Model  Integration  (GATMI)  as  a  model  of  best  practices.  Action  plans  were  created 
based  on  weaknesses  identified  in  the  SGAMPI  B  appraisals  and  were  used  to  baseline  and  track  progress  on  the  imple¬ 
mentation  of  process  improvements.  A  series  of  these  SGAMPI  B  appraisals  was  followed  by  a  SGAMPI  A  appraisal,  in 
June  2006,  when  the  309th  SMKG  was  awarded  a  GMPMI  Maturity  Fevel  5  rating.  PPetails  on  the  strategy  used  and 
lessons-learned  are  shared. 


Anyone  with  experience  in  carpentry 
has  at  some  time  made  a  mistake  and 
cut  a  piece  of  wood  too  short  for  its 
intended  purpose.  What  happens  then? 
After  some  initial  words  of  frustration, 
the  piece  is  either  scrapped,  saved  to  be 
used  for  another  part  that  is  smaller,  or 
the  finished  product  becomes  smaller  in 
one  dimension  than  initially  planned. 

I  do  a  little  woodworking  as  a  hobby. 
Over  the  years,  my  dad  has  been  my  chief 
mentor.  One  of  the  lessons  that  he  taught 
me  —  which  I  will  never  forget  —  is  to 
measure  and  mark  very  carefully  before 
cutting.  I  have  found  that  it  is  a  good  idea 
to  double  (or  even  triple)  check  your  mea¬ 
surements  and  markings  to  make  sure 
that  they  are  correct.  When  I  have  made 
the  mistake  of  cutting  a  piece  too  long,  I 
feel  fortunate  because  at  least  I  get  anoth¬ 
er  chance  to  cut  it  correctly  the  second 
time.  Often  though,  I  make  the  mistake 
of  cutting  a  piece  too  short.  When  this 
happens,  my  dad  will  repeat  an  old  adage 
that  is  simple  to  remember:  Measure  twice, 
cut  once.  These  words  always  remind  me  to 
take  extra  care  to  follow  some  simple 
practices  that  help  me  make  fewer  mis¬ 
takes. 

You  might  say  that  the  same  principle 
applies  to  SCAMPI  A  appraisals  in  an 
organization.  The  time,  effort,  and  cost 
associated  with  such  an  appraisal  are  sub¬ 
stantial.  Before  going  through  with 
SCAMPI  A,  you  want  to  have  some  con¬ 
fidence  that  you  will  meet  the  goals  that 
you  have  for  the  appraisal.  You  do  not 
want  to  go  through  the  trouble  of 
preparing  for  and  holding  a  SCAMPI  A 
unless  you  have  some  confidence  that  the 
goals  of  the  appraisal  will  be  met.  The 
way  to  do  that  is  to  measure  the  readiness 
of  the  organization  over  a  period  of 
time.  There  are  without  a  doubt  different 
ways  that  organizational  readiness  can  be 


measured.  We  as  an  organization  thought 
quite  a  bit  about  how  to  prepare  the  orga¬ 
nization  for  a  SCAMPI  A  and  came  to 
some  conclusions  based  on  lessons 
learned  from  other  organizations,  and 
also  from  our  own  past  experiences. 

Background 

Why  o  SC>!\A1P/  k  Appraisal? 

So  why  did  we  decide  that  we  wanted  to 
have  a  SCAMPI  A  appraisal?  Well,  that  ques¬ 
tion  begs  some  explanation  and  provokes  a 
Mtde  history  lesson. 

In  2002,  with  the  Software 
Engineering  Institute  (SEI)  having  already 
announced  the  future  sunset  of  the  CMM 
and  the  CMM  Based  Assessment  for 
Internal  Process  Improvement  (CBA-IPI) 
method,  the  309th  SMXG  made  a  deci¬ 
sion  to  transition  from  the  CMM  to  the 
newer  CMMI  model.  The  organization 
had  been  rated  CMM  Maturity  Level  5  in 
1998,  and  it  was  time  to  have  a  formal  re¬ 
appraisal  with  the  CMM.  But  rather  than 
making  a  substantial  investment  in  a  CBA- 
IPI  and  then  moving  on  to  the  CMMI,  it 
was  decided  that  the  organizational  strate¬ 
gy  would  be  to  transition  to  the  new 
model  with  the  goal  of  achieving  Level  5 
in  a  couple  of  years. 

As  an  organization,  the  SMXG  has  a 
set  of  strategic  plans  and  goals  that  have 
been  established  by  our  senior  manage¬ 
ment  with  widespread  input  from  below. 
One  of  the  goals  established  was  for  the 
organization  to  be  appraised  at  Maturity 
Level  5  in  the  CMMI.  This  goal  was 
linked  to  several  other  goals  in  areas  such 
as  improvement  in  cost  and  schedule  per¬ 
formance,  quality  improvement,  organi¬ 
zational  growth,  business  development, 
facilities  expansion  and  improvement,  etc. 
For  them,  maintaining  Maturity  Level  5 
was  a  strategic  investment  because  it 
ensured  that  they  were  following  recog¬ 


nized  best  practices,  that  they  were  con¬ 
tinuously  focusing  on  improvement,  and 
that  their  current  and  potential  customers 
could  have  confidence  that  projects  will 
be  planned,  executed,  and  monitored 
properly. 

Transitioning  to  the  CMMI 

The  transition  plan  from  CMM  to  the 
CMMI  for  the  organization  was  mapped 
out  into  three  phases:  Organization 
Documentation  and  Implementation, 
Product-Line  Documentation  and 
Implementation,  and  Appraise  Maturity 
(see  Figure  1). 

Phase  I :  Organization  Documentation 
and  Implementation 

The  goal  of  this  first  phase  was  to  do  the 
following:  Review  the  current  policy, 
plans,  and  processes  used  within  the  orga¬ 
nization  to  determine  where  changes 
needed  to  be  made  to  ensure  compliance 
with  the  CMMI;  update  the  applicable 
documents  as  necessary;  and  ensure 
implementation  at  the  organization  level. 

Process  Action  Teams  (PATs)  were 
established  and  worked  for  approximate¬ 
ly  eight  months,  defining  issues  and  find¬ 
ing  solutions.  The  decision  to  use  PATs 
was  made  in  order  to  get  involvement  and 
buy-in  to  the  needed  changes  from  a 
broad  cross-section  of  the  organization. 
Four  PATs  were  created,  one  for  each  cat¬ 
egory  of  Process  Areas  (PAs)  in  the 
CMMI:  Process  Management,  Project 
Management,  Engineering,  and  Support. 
These  categories  (used  in  the  continuous 
representation  of  the  CMMI)  were  con¬ 
venient  for  partitioning  the  work  that 
needed  to  be  done  by  the  PATs  because 
of  the  common  themes  and  threads  with¬ 
in  each  set  of  PAs  (even  though  they  were 
using  the  staged  representation).  Each 
PAT  had  between  six  and  10  people 
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Figure  1 :  CMM  to  CMMl  Transition  Strategy 


assigned  from  the  ranks  of  projects,  qual¬ 
ity  assurance  (QA),  the  Software 
Engineering  Process  Group  (SEPG),  and 
management.  All  of  these  personnel  were 
working  on  this  task  part-time,  in  addi¬ 
tion  to  their  regular  duties.  At  the  conclu¬ 
sion  of  this  phase,  the  PATs  submitted 
change  proposals  to  the  executive  board 
of  the  organization  for  approval  before 
implementation.  The  executive  board  and 
their  technical  advisors  were  responsible 
to  ensure  that  recommendations  and 
approaches  were  sound  and  consistent 
and  to  give  final  approval  for  implemen¬ 
tation.  Where  conflicts  existed  between 
the  recommendations  of  the  PATs  or 
changes  were  not  accepted,  the  PATs 
were  given  assignments  to  work  out  these 
issues.  Correction  of  these  conflicts 
spanned  nearly  two  months. 

Phase  2:  Product-Line  Documentation 
and  Implementation 

Once  organization-level  policies  and 
processes  were  in  place,  they  needed  to  be 
implemented  at  the  product  line  and  pro¬ 
ject  level.  This  phase  was  similar  in  many 
ways  to  Phase  1  except  for  the  level  at 
which  implementation  was  focused.  PATs 
were  established  in  the  product  lines  and 
included  individuals  who  had  participated 
in  defining  policy  and  processes  at  the 
organizational  level  in  Phase  E  This 
ensured  that  the  people  who  represented 
the  product  line  in  the  organizational- 
level  PATs  were  there  to  assist  the  prod¬ 
uct-line  PATs  in  interpreting  the  changes. 

The  product-line  PATs  had  to  review 
their  processes  to  find  where  gaps  had 
been  created  by  changes  at  the  higher 
level,  to  find  solutions  to  the  gaps  in  the 
context  of  their  business  environment,  to 
implement  these  in  their  processes,  and  to 
train  their  personnel  on  the  changes.  In 
many  ways,  this  proved  to  be  more  diffi¬ 
cult  than  the  prior  phase  because  it  is  usu¬ 
ally  at  this  level  that  the  rubber  meets  the 
road^  so  processes  had  to  be  written  and 
packaged  in  a  way  that  made  sense  in  the 
product  lines.  It  was  also  more  difficult  to 
implement  practices  at  this  level  because 
of  the  large  numbers  of  people  that  had 
to  train  on,  had  to  buy  into,  and  had  to 
start  following  these  practices. 

Phase  3:Appraise  Maturity 

In  this  phase,  the  goals  were  to  accom¬ 
plish  the  following: 

1.  Look  closely  at  what  had  been  imple¬ 
mented,  make  some  judgments  about 
how  well  the  organization  was  satisfy¬ 
ing  CMMI  practices,  and  provide 
some  feedback  to  the  organization 
that  could  be  used  to  make  correc¬ 


tions  where  practices  were  not  imple¬ 
mented  sufficiently. 

2.  When  ready,  hold  a  formal  appraisal 
where  rating  of  a  maturity  level  in  the 
CMMI  could  be  made.  The  tools  we 
decided  to  use  to  meet  these  goals 
were  a  series  of  SCAMPI  B  appraisals, 
followed  by  a  SCAMPI  A  appraisal. 
Within  this  phase,  they  implemented  a 
plan  for  how  the  appraisals  would  be 
structured.  You  could  say  that  the 
Appraise  Maturity  phase  was  made  up  of 
three  rounds  of  appraisals.  The  first 
round  was  to  use  the  SCAMPI  B  method 
and  was  intended  to  baseline  where  the 
organization  and  the  projects  being 
looked  at  were  at  that  point  in  time.  It 
provided  a  basis  for  the  initial  action 
plans.  The  second  round  of  appraisals 
was  also  to  use  SCAMPI  Bs,  but  this 
round  was  intended  to  measure,  in  an 
appraisal  environment,  how  much 
improvement  had  taken  place  at  approxi¬ 
mately  the  mid-point  of  the  Appraise 
Maturity  phase.  The  third  round  in  this 
phase  was  the  SCAMPI  A  appraisal. 

The  SMXG  decided  to  select  a  num¬ 
ber  of  projects  that  would  represent  the 
organization  well  in  terms  of  workload 
performed  and  in  terms  of  numbers  of 
people  assigned  to  them.  Other  projects 
in  the  organization  that  were  not  included 
in  the  SCAMPI  B  appraisals  were  not 
ignored.  They  were  expected  to  imple¬ 
ment  the  same  changes  as  projects 
involved  in  the  appraisals;  the  mechanism 
being  used  to  monitor  their  progress  was 


done  using  our  standard  internal  QA 
audit  function. 

After  much  thought  and  discussion, 
they  settled  on  six  projects  to  participate 
in  the  SCAMPI  Bs.  The  projects  included 
in  these  appraisals  were  typically  the  larg¬ 
er  and  most  mainstream  in  the  organiza¬ 
tion.  The  same  set  of  six  projects  was  to 
be  examined  in  both  of  the  first  two 
appraisal  rounds. 

Overlapping  Phases 

The  three  phases  of  the  transition  strate¬ 
gy  overlapped  each  other  as  shown  in 
Figure  1.  The  phases  were  not  a  true 
waterfall  model  where  each  phase  would 
be  completed  before  moving  on  to  the 
next  one.  When  the  product  lines  started 
tailoring  their  processes  to  be  compliant 
with  new  policy  and  organizational-level 
processes,  it  sometimes  pointed  out  prob¬ 
lems  with  these  higher-level  documents 
that  needed  to  be  addressed.  This 
required  that  activities  be  done  in  Phase  1 
again.  Likewise,  when  appraisals  were 
performed,  it  pointed  out  cases  where 
organization,  product  line,  or  project 
approaches  or  documents  were  deficient 
and  needed  correction,  and  required  re¬ 
entry  into  the  prior  two  phases. 

Collaborative  Approach  to 
SCAMPI  B 

A  couple  of  years  before  the  expected 
date  of  the  SCAMPI  A  appraisal,  the 
SMXG  selected  a  lead  appraiser  and 
began  to  talk  with  him.  The  lead  apprais- 
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er  who  they  chose  was  Dr.  Miluk  from 
the  SEL  In  the  initial  discussions,  Miluk 
told  them  about  some  recent  experiences 
leading  what  he  called  collaborative 
SCAMPI  B  appraisals  with  organizations 
within  some  large  corporations  in  the 
defense  and  commercial  communications 
industries,  as  well  as  with  the  SMXG’s  sis¬ 
ter  organization  at  Warner-Robins  Air 
Logistics  Center,  the  402d  SMXG.  He 
told  us  how  this  new  approach  to 
SCAMPI  B  appraisals  had  worked  well  to 
not  only  identify  areas  of  strength  and 
weakness  with  respect  to  the  model,  but 
had  also  benefited  the  organization  in 
other  ways.  Having  had  much  experience 
in  CMM  and  CMMI  appraisals  in  the  past 
themselves,  they  knew,  like  Miluk,  that 
the  appraisal  results  often  are  pretty  clear 
to  the  SEPG  and  others  well  initiated  in 
process  improvement  but  are  often  con¬ 
fusing  to  lay  people.  Miluk  explained  that 
this  new  approach  helps  in  several  ways, 
including  the  following:  provides  projects 
with  a  better  understanding  of  what  the 
model  is  asking  them  to  do;  makes  it 
more  clear  to  projects  what  their  weak¬ 
nesses  are  and  how  to  fix  them;  receives 
buy-in  from  key  project  personnel  that 
weaknesses  identified  in  the  appraisal 
were  valid.  After  much  thought  and  dis¬ 
cussion  with  senior  management,  they 
decided  to  go  ahead  and  implement  this 
approach. 

Because  they  were  going  to  be  con¬ 
ducting  a  string  of  SCAMPI  B  appraisals 
over  a  period  of  12-18  months,  and  since 
they  had  a  number  of  authorized 
SCAMPI  lead  appraisers  within  the  orga¬ 
nization,  they  asked  Miluk  to  lead  the  first 
SCAMPI  B  appraisal  to  train  them  on  the 
collaborative  approach.  From  there,  they 
had  resources  available  internally  to  lead 
the  rest  of  the  SCAMPI  Bs. 

Project  and  Organization  Focused 
Mini-Teams 

In  SCAMPI  appraisals,  the  work  of 
examining  artifacts  and  the  responsibility 
of  making  preliminary  judgments  about 
the  degree  of  CMMI  model  compliance 
is  typically  distributed  among  groups  of 
two  or  three  appraisal  team  members. 
These  groups  of  appraisal  team  members 
are  called  mini-teams.  The  use  of  mini¬ 
teams  allows  for  increased  efficiency  in 
the  appraisal  process. 

One  of  the  most  important  elements 
in  the  design  of  their  collaborative 
SCAMPI  B  appraisals  was  to  align  the 
mini-teams  primarily  by  project  and  to 
make  sure  that  key  project  personnel  were 
included  in  the  mini-team  looking  at  their 
project.  Because  a  whole  series  of 


appraisals  were  happening  in  a  relatively 
short  period  of  time,  there  was  no  need  to 
include  organizationally  focused  PAs  from 
the  CMMI  in  all  of  the  appraisals.  This 
inclusion  would  have  resulted  in  unneces¬ 
sary  appraisal  redundancy;  these  PAs  were 
examined  in  just  two  of  the  SCAMPI  Bs. 
In  these  two  particular  SCAMPI  Bs,  they 
had  a  single  mini-team  focused  on  these 
PAs  (e.g.  Organizational  Process  Focus, 
Organizational  Process  Definition, 
Organizational  Training  etc.).  The  number 
of  projects  examined  in  each  SCAMPI  B 
varied,  with  the  maximum  being  three 
projects  plus  the  organization-level 
processes  totaling  four  mini-teams. 

Competencies 

In  determining  the  appraisal  team  mem¬ 
bers,  the  following  three  critical  compe¬ 
tencies  were  required  for  each  mini-team: 

1.  Project  knowledge.  It  was  impor¬ 
tant  to  have  someone  on  each  mini¬ 
team  who  knew  where  to  look  for 

The  use  of  mini-teams 
allows  for  increased 
efficiency  in  the 
appraisal  process. 

necessary  artifacts  when  the  provided 
ones  were  insufficient.  Additionally, 
they  wanted  to  have  someone  there 
who  could  explain  in  finer  detail,  if 
needed,  the  explanation  for  each 
practice  provided  in  the  program 
independent  interfaces  (PIIs).  This 
competency  was  met  using  the  pro¬ 
ject  manager  (PM)  or  a  project  lead 
engineer. 

2.  Process  improvement  experience 
in  the  organization.  The  impor¬ 
tance  of  this  competency  is  in  maxi¬ 
mizing  buy-in  to  the  appraisal  find¬ 
ings.  In  the  organization,  the  lead 
process  improvement  agents  in  the 
product  lines  are  called  Senior 
Technical  Program  Managers 
(TPMs).  They  are  skilled,  experienced 
PMs  who  have  responsibility  to  men¬ 
tor  and  assist  the  managers  of  pro¬ 
jects  within  their  product  lines,  along 
with  leading  process  improvement 
efforts  at  that  level  of  the  organiza¬ 
tion.  The  SMXG  decided  to  use  these 
individuals  to  help  meet  this  compe¬ 
tency  level  because  they  knew  that 
having  their  support  was  essential  and 
that  making  them  part  of  the  teams 


on  these  SCAMPI  B  appraisals  would 
help  to  ensure  ownership  of  the 
appraisal  findings  and  actions  needed 
to  address  them. 

3.  Appraisal  and  CMMI  model 
expertise.  Each  mini-team  had  an 
individual  who  was  either  a  lead 
appraiser  or  had  significant  experience 
in  appraisals,  as  well  as  interpreting 
and  implementing  the  CMMI.  This 
competency  was  addressed  by  using 
individuals  who  were  members  of  the 
SEPG  (who  are  lead  appraisers),  the 
internal  QA  group,  or  lead  appraisers 
from  the  Software  Technology  Sup¬ 
port  Center  (STSC). 

Project-Focused  Draft  and  Final 
Findings  Briefings 

Another  important  feature  in  the  design  of 
the  collaborative  SCAMPI  Bs  was  to  pro¬ 
vide  draft  and  final  findings  briefings  at  the 
project  level  to  make  sure  identified  weak¬ 
nesses  were  clearly  understood.  The  pro¬ 
ject  leaders  included  in  the  appraisal  team 
were  given  a  conspicuous  role  in  briefing 
the  results  of  their  projects  as  a  means  to 
add  legitimacy  to  the  findings  and  to  help 
secure  buy-in.  In  some  cases,  the  appraisal 
team  members  representing  the  project 
were  the  presenters  in  the  findings  briefin¬ 
gs.  Where  necessary,  these  individuals 
could  explain  to  personnel  from  their  own 
projects  what  the  weaknesses  meant,  could 
discuss  why  correcting  these  weaknesses 
would  add  value  to  the  project,  and  also 
could  discuss  with  the  PM  ways  to  address 
and  correct  the  issue. 

After  Each  Appraisal 

Now  that  a  particular  SCAMPI  B 
appraisal  was  over,  the  real  work  of 
addressing  issues  and  implementing 
changes  was  about  to  begin.  One  of  the 
things  that  made  our  efforts  to  reach 
CMMI  Level  5  more  difficult  than  for 
other  organizations  was  our  size  and  rela¬ 
tive  diversity  in  our  product  line  make-up. 

This  made  our  action  planning  and 
tracking  of  these  plans  critical.  In  our 
case,  the  implementation  effort  was 
focused  more  at  the  lower  levels  of  the 
organization  (product  line  and  project 
levels)  where  more  of  our  weaknesses 
were.  The  real  challenge  at  the  organiza¬ 
tion  level  was  making  sure  that  action 
plans  were  complete  and  were  monitored 
to  ensure  that  items  would  be  completed 
in  order  to  meet  our  timeframe  goals  for 
the  SCAMPI  A  appraisal. 

Action  Planning 

Plans  for  addressing  appraisal  weaknesses 
were  created  at  the  product-line  level,  and 
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also  by  the  SEPG  for  organization-level 
weaknesses.  These  plans  were  then 
brought  together  at  the  organization  level 
to  provide  an  opportunity  for  review  by 
all  of  the  stakeholders.  The  plans  includ¬ 
ed  a  work  breakdown  structure  of  tasks, 
personnel  and  resources  assigned  to  each, 
effort  estimates,  schedules,  risks  (and  mit¬ 
igation  plans),  etc. 

Tracking  Progress 

In  tracking  progress  of  the  action  plans, 
they  used  the  following  four  types  of 
charts:  1)  action  plan  activity  descriptions, 
2)  Gantt  charts,  3)  percent  complete 
charts,  and  4)  charts  that  showed  our  cur¬ 
rent  estimated  risk  of  satisfying  each 
practice  in  the  CMMI  based  on  the  char¬ 
acterizations  used  in  SCAMPI  B 
appraisals  (High  [Red],  Medium  [Yellow], 
Low  [Green])  as  shown  in  Figure  2.  After 
a  short  time,  chart  type  4  was  given  the 
moniker  The  Red-Green  chart  after  the 
character  from  the  Canadian  television 
show  (The  Red  Green  Show)  that  is  shown 
here  in  the  U.S.  on  the  Public 
Broadcasting  Service.  These  were  briefed 
every  two  weeks  to  senior  management 
and  provided  them  with  detailed  informa¬ 
tion  about  what  issues  were  being  worked 
and  were  scheduled  to  be  worked,  prob¬ 
lems  that  were  being  encountered,  and 
how  well  the  tasks  were  being  completed 
compared  to  the  schedule. 

In  order  to  change  the  risk  characteri¬ 
zation  on  any  practice  on  the  Red-Green 
chart,  a  project  had  to  have  their  remedy 
to  the  identified  weakness  reviewed  by 
the  SEPG  (who  have  extensive  CMMI 
model  interpretation  and  appraisal  experi¬ 
ence)  as  a  QA  check.  Only  actions  that 
could  show  artifacts  which  satisfied  a  par¬ 
ticular  practice  were  accepted  toward 
changing  a  Red  (High  risk)  or  Yellow 
(Medium)  to  Green  (Low).  A  partial  solu¬ 
tion  to  a  weakness  would  permit  a  change 
in  the  Red-Green  chart  from  Red  to  Yellow. 

The  SCAMP!  A 

The  organization  had  made  a  goal  to  hold 
the  SCAMPI  A  appraisal  late  in  2005. 
This  goal  proved  to  be  a  bit  too  aggres¬ 
sive.  Some  funding  constraints  were 
levied  on  them  from  the  Air  Force  in 
mid-2005  that  limited  their  ability  for 
Process  Action  Teams  (PATs)  to  function 
at  the  pace  required  to  address  all  the 
issues  that  needed  to  be  solved  in  that 
timeframe.  So,  the  goal  for  holding  the 
SCAMPI  A  was  pushed  back  to  June 
2006.  That  proved  to  be  a  good  thing  in 
the  end  because  it  provided  more  time  to 
do  things  right  and  to  institutionalize 
changes.  It  also  made  efforts  to  prepare 


for  the  SCAMPI  A  a  lot  less  pressured 
than  if  we  had  continued  the  march 
toward  the  original  goal. 

The  SCAMPI  A  appraisal  was  per¬ 
formed  according  to  requirements  of  the 
method.  Some  of  the  choices  that  we 
made  in  executing  the  appraisal  were  the 
following: 

1.  Selection  of  focus  projects.  Focus 
projects  were  chosen  from  each  of 
the  product  lines  in  the  organization. 
Projects  chosen  were  representative 
of  the  majority  of  work  performed  in 
the  product  lines  and  represented  41 
percent  of  the  engineering  personnel 
in  the  entire  309th  SMXG 

2.  Appraisal  team  size  and  member¬ 
ship.  The  appraisal  team  had  12  mem¬ 
bers  with  three  members  per  mini¬ 
team.  This  is  probably  larger  than  the 
average  for  a  SCAMPI  A  appraisal. 
The  lead  appraiser  assigned  a  repre¬ 
sentative  from  each  of  the  organiza¬ 
tion’s  product  lines,  a  person  with  lead 
appraiser  credentials  —  or  a  very  expe¬ 
rienced  appraisal  veteran  —  and  an  indi¬ 
vidual  from  an  outside  government 
organization  with  high-maturity  expe¬ 
rience  to  each  mini-team. 

The  members  of  the  appraisal 
team  from  outside  the  organization 
were  involved  in  one  of  the  earlier 
SCAMPI  B  appraisals.  This  helped 
them  to  come  up- to- speed  on  the 
structure  of  the  organization  and  its 
product  lines,  and  it  familiarized  them 
with  some  of  the  types  of  projects 
that  exist  in  309  SMXG  and  the 
processes  used. 

3.  An  expanded  readiness  review.  A 

readiness  review  was  held  approxi¬ 
mately  four  weeks  prior  to  the 
planned  start  of  the  on-site  period. 
The  SCAMPI  A  method  requires  that 
the  readiness  review  includes  a  review 


of  the  artifacts  (or  PIIs)  and  the  plans 
to  make  sure  that  these  are  in  order 
and  that  the  appraisal  has  a  reason- 
able-to-high  probability  of  being 
completed  according  to  plan.  The 
readiness  review  included  these  activi¬ 
ties  but  in  addition,  a  very  detailed 
examination  of  the  artifacts  was  per¬ 
formed.  The  readiness  review  lasted 
five  full  days  and  included  some 
method  training  and  a  full  review  of 
artifacts  in  the  PIIs  where  preliminary 
characterizations  were  made  on  the 
instantiations  examined.  At  the  con¬ 
clusion  of  the  readiness  review,  the 
appraisal  sponsor  was  given  a  list  of 
additional  information  and  artifacts 
needed  along  with  the  lead  appraiser’s 
assessment  of  the  readiness  to  pro¬ 
ceed  to  the  on-site.  The  green  light  to 
proceed  was  given. 

4.  Mini-team  organization.  Mini¬ 
teams  were  assigned  to  examine  all  the 
non-organizational  project-oriented 
PAs  for  a  particular  project,  as  had 
been  done  in  the  prior  SCAMPI  Bs. 
They  found  that  for  this  organization 
this  was  the  most  reasonable 
approach.  Another  mini-team  was 
assigned  to  examine  the  PA’s  that  had 
an  organizational  focus. 

5.  Interview  Sessions.  The  first  week 
of  the  on-site  period  was  spent  entire¬ 
ly  on  briefings,  interview  sessions  with 
the  focus  projects,  tool  demonstra¬ 
tions,  and  additional  method  training 
(where  needed).  During  the  early  part 
of  the  second  week,  another  set  of 
interview  sessions  was  held  with  func¬ 
tional  representatives  of  a  much  wider 
group  of  additional  projects.  This  was 
done  to  enrich  the  sampling  within 
the  organization  to  ensure  consistency 
and  institutionalization  of  CMMI 
model  practices. 


Figure  2:  Red-Green  Ghart  Rxample 
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Lessons  Learned 

The  collaborative  approach  to  SCAMPI  B 
appraisals  was  very  effective  for  the 
SMXG.  It  met  the  expectations  that  they 
had  in  terms  of  improving  the  ability  of 
the  appraisal  team  to  find  appropriate  arti¬ 
facts  when  the  PIIs  provided  were  less 
than  perfect.  This  approach  also  improved 
buy-in  to  the  weaknesses  identified  in  the 
appraisals.  The  use  of  project  leaders  on 
mini-teams  improved  the  ability  of  the 
project  leadership  to  understand  the  con¬ 
cepts  behind  model  practices  and  facilitat¬ 
ed  their  commitment  to  implementing 
value-added  improvements  to  address 
these  weaknesses.  As  an  organization,  they 
have  some  experience  using  informal 
appraisals  to  prepare  for  formal  appraisals 
or  in  piloting  new  models  and  definitely 
feel  that  this  strategy  works  and  this 
approach  will  be  used  again  in  the  future. 

The  management  of  action  plans 
using  Gantt,  Percent- Complete,  and  Red- 
Green  charts  in  concert  with  Current 
Activity  description  slides  was  critical  for 
SMXG  in  getting  this  large  effort  com¬ 
pleted.  Had  it  not  been  for  the  clear  mea¬ 
sures  and  frequent  reporting,  the  goal  of 
achieving  CMMI  Level  5  would  not  have 
been  met,  at  least  not  in  the  timeframe 
that  they  had  in  mind. 

The  SMXG  found  that  it  was  critical  to 
have  QA  on  the  closeout  of  action  items, 
and  the  changing  of  risk  characterizations 
that  were  represented  on  the  Red-Green 
charts.  Without  this  step,  they  would  cer¬ 


tainly  have  had  some  things  that  were 
thought  to  be  sufficiently  addressed  that 
would  have  come  back  again  as  a  weak¬ 
ness  in  a  subsequent  appraisal. 

PII  preparation  and  maintenance 
through  the  string  of  appraisals  that 
SMXG  performed  turned  out  to  be  a 
major  issue.  Projects  who  took  part  in  the 
appraisals  were  responsible  to  prepare 
their  own  PIIs,  and  some  were  more  effi¬ 
cient  and  skilled  than  others.  The  effort 
and  cost  of  preparing  PIIs  was  excessive 
through  the  Appraise  Maturity  phase.  This 
is  one  area  that  they  will  be  focusing  some 
innovation  on  to  improve  efficiency  and 
reduce  cost.  They  have  some  ideas  about 
potential  tools  and  methods  of  leveraging 
from  the  internal  QA  audits  and  data  map¬ 
pings  to  help  in  preparing  PIIs  and  plan  to 
pursue  these  ideas  over  the  next  year  or  so. 

Conclusion 

The  adoption  and  institutionalization  of 
the  CMMI  in  the  309th  SMXG  was  a 
journey  that  lasted  approximately  four 
years.  It  was  a  goal  that  would  not  have 
been  reached  without  the  continual  sup¬ 
port  of  aU  levels  of  management  and  the 
innovation  and  hard  work  of  many  indi¬ 
viduals  from  aU  levels  of  the  organiza¬ 
tion.  The  collaborative  appraisal  concept 
for  SCAMPI  B  was  a  critical  factor  in 
transferring  understanding  of  CMMI 
practices  to  projects  and  in  obtaining 
buy-in  to  implementing  them  in  a  value- 
added  manner.  ♦ 
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